NASA Technical Reports Server (NTRS) 19910007721: Advanced training systems

PRESENTATION 3.1.2

N91 - 17034

ADVANCED TRAINING SYSTEMS

423

SPACE TRANSPORTATION AVIONICS
TECHNOLOGY SYMPOSIUM

WILLIAMSBURG, VIRGINIA
NOVEMBER 7-9, 1989

ADVANCED TRAINING SYSTEMS
WHITE PAPER

Robert T. Savely
Software Technology Branch
NASA/Johnson Space Center
Houston, Texas
713-483-8105

R. Bowen Loftin, Ph.D.
University of Houston-Downtown
Houston, Texas
713-483-8070

PRECEDING PAGE BLANK NOT FILMED

425

ADVANCED

TRAINING SYSTEMS

Robert T. Savely R. Bowen Loft in, Ph.D.

Software Technology Branch University of Houston-Downtown

NASA/Johnson Space Center Houston, Texas

Houston, Texas 713-483-8070

713-483-8105

Space Transportation Avionics Technology Symposium
Williamsburg, Virginia
November 7-9, 1989

I. INTRODUCTION

A. General Introduction

Training is a major endeavor in all modern societies: new
personnel must be trained to perform the task(s) which they were
hired to perform, continuing personnel must be trained to upgrade
or update their ability to perform assigned tasks, and continuing
personnel must be trained to tackle new tasks. Common methods
include training manuals, formal classes, procedural computer
programs, simulations, and on-the-job training. The latter method
is particularly effective in complex tasks where a great deal of
independence is granted to the task performer. Of course, this
training method is also the most expensive and may be impractical
when there are many trainees and few experienced personnel to
conduct on-the-job training.

NASA's training approach has focussed primarily on on-the— job
training in a simulation environment for both crew and ground-
based personnel. This process worked relatively well for both the
Apollo and Space Shuttle programs. Space Station Freedom and
other long range space exploration programs coupled with limited
resources dictate that NASA explore new approaches to training for
the 1990 's and beyond.

This report describes specific autonomous training systems
based on artificial intelligence technology for use by NASA
astronauts, flight controllers, and ground-based support personnel
that demonstrate an an alternative to current training systems.
In addition to these specific systems, the evolution of a general
architecture for autonomous intelligent training systems that
integrates many of the features of "traditional" training programs
with artificial intelligence techniques is presented. These
Intelligent Computer-Aided Training (ICAT) systems would provide,
for the trainee, much of the same experience that could be gained
from the best on-the-job training. By integrating domain
expertise with a knowledge of appropriate training methods, an
ICAT session should duplicate, as closely as possible, the trainee
undergoing on-the-job training in the task environment, benefiting
from the full attention of a task expert who is also an expert
trainer. Thus, the philosophy of the ICAT system is to emulate

426

the behavior of an experienced individual devoting his full time
and attention to the training of a novice — proposing challenging
training scenarios, monitoring and evaluating the actions of the
trainee, providing meaningful comments in response to trainee
errors, responding to trainee requests for information, giving
hints (if appropriate), and remembering the strengths and
weaknesses displayed by the trainee so that appropriate future
exercises can be designed.

B . BACKGROUND

Since the 1970's a number of academic and industrial
researchers have explored the application of artificial
intelligence concepts to the task of teaching a variety of
subjects [Sleeman and Brown, 1982; Yazdani, 1986; Wenger, 1987]
(e.g., computer programming in Lisp (Anderson, 1985; Anderson,
Boyle and Reiser, 1985] and Pascal [Johnson and Soloway, 1985],
economics [Shute and Bonar, 1986], geography [Carbonell, 1970],
and geometry [Anderson, Boyle and Yost, 1985]). The earliest
published reports which suggested the applications of artificial
intelligence concepts to teaching tasks appeared in the early
1970's [Carbonell, 1970; Hartley and Sleeman, 1973]. Hartley and
Sleeman [Hartley and Sleeman, 1973] actually proposed an
architecture for an intelligent tutoring system. However, it is
interesting to note that, in the sixteen years which have passed
since the appearance of the Hartley and Sleeman proposal, no
agreement has been reached among researchers on a general
architecture for intelligent tutoring systems [Yazdani, 1986] .

Along with the extensive work on intelligent tutoring systems
for academic settings has come the development of systems directed
at training. Among these are Recovery Boiler Tutor [Woolf,
Blegen, Jansen, and Verloop, 1986], SOPHIE [Brown, Burton and de
Kleer, 1982], and STEAMER [Hollan, Hutchins and Weitzman, 1984].
These differ from the tutoring systems mentioned above in
providing a simulation model with which the student or trainee
interacts. Although these intelligent training systems each use
the interactive simulation approach, they each have very different
internal architectures. Further, there appears to be no
agreement, at present, on a general architecture for such
simulation training systems. The work reported here builds on
these previous efforts and our own work [Loftin, Wang, Baffes and
Rua, 1987; Loftin, Wang, Baffes, and Hua, 1988; Loftin, Wang,
Baffes, and Hua, 1989a and b] to develop specific intelligent
training systems as well as a general approach to the design of
intelligent training systems which will permit the production of
such systems for a variety of tasks and task environments with
significantly less effort that is now required to ''craft" such a
system for each application.

C. TRAINING VERSUS TUTORING

The ICAT systems and architecture described here have been
developed with a clear understanding that training is not the same

427

as teaching or tutoring [Harmon, 1987] . An industrial or
governmental training environment differs in many ways from an
academic teaching environment These differences are important in
' the design of an architecture for an intelligent training system:

• Assigned tasks are often mission-critical, placing the
responsibility for lives and property in the hands of those
who have been trained.

• Personnel may already have significant academic and
practical experience to bring to bear on their assigned
task .

• Trainees make use of a wide variety of training techniques,
ranging from the study of comprehensive training manuals to
simulations to actual on-the-job training under the
supervision of more experienced personnel.

• Many of the tasks offer considerable freedom in the exact
manner in which they may be accomplished.

Those undergoing training for complex tasks are usually well
aware of the importance of their job and the probable consequences
of failure. While students are often motivated by the fear of
receiving a low grade, trainees know that human lives and/or
expensive equipment may depend on their skill in performing
assigned tasks. This means that trainees may be highly motivated,
but it also imposes on the trainer the responsibility for the
accuracy of the training content (i.e., verification of the domain
expertise encoded in the system) and the ability of the trainer to
correctly evaluate trainee actions. The ICAT approach is
intended, not to impart basic knowledge, but to aid the trainee in
developing skills for which he already has the basic or
"theoretical" knowledge. In short, this training system
architecture is designed to help a trainee put into practice that
which he already intellectually understands. The system must take
into account the type of training that both precedes and follows,
building on the knowledge gained from training manuals and rule
books while preparing the trainee for and complementing the on-
the-job training which may follow. Perhaps most critical of all,
trainees must be allowed to carry out an assigned task by any
valid means. Such flexibility is essential so that trainees are
able to retain, and even hone, an independence of thought and
develop confidence in their ability to respond to problems, even
problems which they have never encountered and which their
trainers never anticipated.

IV. APPLICATIONS

The ICAT architecture was originally applied to a training
system for NASA flight controllers learning to deploy satellites
from the Space Shuttle. The same architecture has been used in
the construction of ICAT systems for training astronauts for
SpaceLab missions and engineers who test the Space Shuttle main

428

propulsion system. Although these tasks are quite different and
are performed in very dissimilar environments, the same system
architecture has proven to be adaptable to each. Below is a brief
summary of the specific systems that have been built or are
currently under development:

A. PD/ICAT: [£ayload-assist module Deploys/ICAT System]

A comprehensive intelligent computer-aided training system for use
by Flight Dynamics Officers in learning to deploy PAM (Payload-
Assist Module) satellites from the Space Shuttle. PD/ICAT
contains four expert systems that cooperate via a blackboard
architecture .

B. WL/ICAT: [Vacuum Yent Line/ICAT System]

A PC-based intelligent computer-aided training system for use by
mission and payload specialists in learning to perform fault
detection, isolation, and reconfiguration on the Spacelab WL
system. WL/ICAT consists of an integrated expert system and
graphical user interface.

C. MPP/ICAT: [Main Propulsion Pneumatics/ICAT System]

A comprehensive intelligent computer-aided training system for use
by test engineers at NASA/Kennedy Space Center in learning to
perform testing of the Space Shuttle Main Propulsion Pneumatics
system. MPP/ICAT is currently under development and makes use of
the same architecture as PD/ICAT.

D. IPS/ICAT: [Instrument Pointing System/ICAT System]

A comprehensive intelligent computer-aided training system for use
by payload and mission specialists at NASA/Johnson Space Center
and Marshall Space Flight Center in learning to utilize the IPS on
Spacelab missions. IPS/ICAT is currently under development and
makes use of the same architecture as PD/ICAT.

III. A GENERAL ARCHITECTURE FOR INTELLIGENT TRAINING SYSTEMS

The projects described in the previous section have served as
vehicles to aid in the design and refinement of an architecture
for intelligent training systems that has significant domain-
independent elements and is generally applicable to training in
procedural tasks common to the NASA environment. The ICAT system
architecture is modular and consists of five basic components:

• A user interface that permits the trainee to access the
same information available to him in the the task
environment and serves as a means for the trainee to take
actions and communicate with the intelligent training
system.

429

• A domain expert which can carry out the task using the same
information that is available to the trainee and which also
contains a list of "mal-rules" (explicitly identified
errors that novice trainees commonly make) .

• A training session manager which examines the actions taken
by the domain expert (of both correct and incorrect actions
in a particular context) and by the trainee and takes
appropriate action(s). [Loftin, Baffes and Wang, 1988]

• A trainee model which contains a history of the individual
trainee's interactions with the system together with
summary evaluative data.

• A training scenario generator that designs increasingly-
complex training exercises based on the knowledge of the
domain expert, the current skill level contained in the
trainee's model, and any weaknesses or deficiencies that
the trainee has exhibited in previous interactions.
[Loftin, Wang, and Baffes, 1988; Loftin, Wang, and Baffes,
1989]

Figure 1 contains a schematic diagram of the ICAT system. Note
that provision is made for the user to interact with the system in
two distinct ways and that a supervisor may also query the system
for evaluative data on each trainee. The blackboard serves as a
common repository of facts for all five system components. With
the exception of the trainee model, each component makes
assertions to the blackboard, and the expert system components
look to the blackboard for facts against which their rules pattern
match. A comprehensive effort has been made to clearly segregate
domain-dependent from domain-independent components.

IV. SYSTEM INTEGRATION

The ICAT architecture described above was originally
implemented in a Symbolics 3600 Lisp environment using Inference
Corporation's ART for the rule-based components. The architecture
is currently available for unix workstations. The user interface
is implemented in X-Windows, the rule-based components in CLIPS
[CLIPS is the acronym for a NASA-developed expert system shell
written in C], and supporting code in C.

V. TRAINING PERFORMANCE

The original system developed with this architecture
(PD / ICAT ) has been used by both expert and novice flight
controllers at NASA/Johnson Space Center. An extensive
investigation of the performance of novices using the system has
been conducted. Figure 2 shows two measures of performance: (1)

the time required to perform the nominal task as a function of the
number of training experiences and (2) the number of errors made
during the performance of the nominal task as a function of the
number of training experiences. It is interesting to note that.

430

although the novices used in this investigation had very different
levels of prior experience related to the task, all novices
rapidly approached the same level of proficiency.

VI. CONCLUSIONS

A general architecture for ICAT systems has been developed
and applied to the construction of three ICAT systems for very
different tasks. Use by novices of an ICAT application built upon
this architecture has shown impressive trainee performance
improvements. With further refinement and extension, this
architecture promises to provide a common foundation upon which to
build intelligent training systems for many tasks of interest to
the government, military, and industry. The availability of a
robust architecture that contains many domain-independent
components serves to greatly reduce the time and cost of
developing new ICAT applications.

REFERENCES

[Anderson and Reiser, 1985] Anderson, J.R. and Reiser, B.J., The
LISP Tutor, Byte, Vol . 10, No. 4, pp. 159-175, April, 1985.

[Anderson, Boyle and Reiser, 1985] Anderson, J.R., Boyle, C.F.,
and Reiser, B.J., Intelligent Tutoring Systems, Science,
Vol. 225, No. 4698, pp. 456-462, 1985.

[Anderson, Boyle and Yost, 1985] Anderson, J . R . , Boyle C. F., and
Yost, G., The Geometry Tutor, Proceedings of the Ninth
International Joint Conference on Artificial Intelligence, Los
Angeles, CA, 1985, pp. 1-7.

[Brown, Burton and de Kleer, 1982] Brown, J.S., Burton, R.R., and
de Kleer, J., Pedagogical, Natural Language and Knowledge
Engineering Techniques in SOPHIE I, II, and III, in Sleeman, D.
and Brown, J.S., (eds.). Intelligent Tutoring Systems, Academic

Press, London, 1982, p. 227.

[Carbonell , 1 970 ] Carbonell, J.R. AI in CAI : An Artificial

Intelligence Approach to CAI, IEEE Transactions on Man-Machine
Systems, Vol. 11, No. 4, pp. 190-202, 1970.

[Harmon, 1987] Hatmon, P. Intelligent Job Aids: How AI Will

Change Training in the Next Five Years, in Kearsley, G., ed.,
Artificial Intelligence and Instruction: Applications and

Methods, Addison Wesley Publishing Co., Reading, MA, 1987.

[Hartley and Sleeman, 1973] Hartley, J.R. and Sleeman, D.H.,
Towards Intelligent Teaching Systems, International Journal of
Man-Machine Studies, Vol. 5, pp. 215-236, 1973.

[Hollan, Hutchins and Weitzman, 1984] Hollan, H.D., Hutchins,
E.L., and Weitzman, L. Steamer: An Interactive Inspectable

431

Simulation-based Training System, AI Magazine, Summer, 1984,
pp . 15-27 .

[Johnson and Soloway, 1985] Johnson, W.L. and Soloway, E. PROUST,
Byte, Vol. 10, No. 4, pp. 179-190, April, 1985.

[Kearsley, 1987] G. Kearsley, ed., Artificial Intelligence and
Instruction, Addison-Wesley, Reading. MA, 1987.

[Loftin, Wang, Baffes and Rua, 1987] Loftin, R.B., Wang, L.,

Baffes, P., and Rua, M., An Intelligent Computer-Aided Training
System for Payload-Assist Module Deploys, Proceedings of the
First Annual Workshop on Space Operations, Automation &
Robotics (SOAR '87), NASA/Johnson Space Center, Houston, TX,
August 4-6, 1987.

[Loftin, Baffes and Wang, 1988] Loftin, R.B., Baffes, P., and

Wang, L, An Approach to the Diagnosis of Trainee Errors in an
Intelligent Training System, unpublished report.

[Loftin, Wang, Baffes, and Hua, 1988] Loftin, R.B., Wang, L.,

Baffes, P., and Hua, G., "An Intelligent Training System for
Space Shuttle Flight Controllers," Informatics and Telematics
5 (3) , 151 (1988) .

[Loftin, Wang, and Baffes, 1988] Loftin, R.B., Wang, L., and

Baffes, P., "Simulation Scenario Generation for Intelligent
Training Systems," Proceedings of the Third Artificial
Intelligence and Simulation Workshop, August 22, 1988, St.

Paul, MN, pp. 69-74.

[Loftin, Wang, Baffes, and Hua, 1989a] Loftin, R.B., Wang, L.,

Baffes, P., and Hua, G., "An Intelligent System for Training
Space Shuttle Flight Controllers in Satellite Deployment
Procedures," to be published in Machine Mediated Learning .

[Loftin, Wang, Baffes, and Hua, 1989b] Loftin, R.B., Wang, L.,

Baffes, P., and Hua, G., "An Intelligent Training System for
Space Shuttle Flight Controllers" in Innovative Applications
of Artificial Intelligence, edited by H. Schorr and A.
Rappaport (Menlo Park, CA: AAAI Press, 1989), pp . 15-24.

[Loftin, Wang, and Baffes, 1989] Loftin, R.B., Wang, L., and

Baffes, P., "Intelligent Scenario Generation for Simulation-
Based Training, " Proceedings of the American Institute for
Aeronautics and Astronautics Computers in Aerospace VII
Conference, Monterey, CA, October 3-5, 1989, pp . 581-588.

[Shute and Bonar, 1986] Shute, V. and Bonar, J.G., An Intelligent
Tutoring System for Scientific Inquiry Skills, Proceedings of
the Eighth Cognitive Science Society Conference Amherst, MA,
1986, pp. 353-370.

432

[ Sleeman and Brown, 1982] D. Sleeman and J.S. Brown, eds.,
Intelligent Tutoring Systems, Academic Press, London, 1982.

[Sleeman, 1982] Sleeman, D.H. Inferring (mal) rules from pupils'
protocols. Proceedings of the European Conference on
Artificial Intelligence, Orsay, France, 1982, pp . 160-164.

[Wenger, 1987] Wenger, E., Artificial Intelligence and Tutoring
Systems, Morgan Kaufmann Publishers, Los Altos, CA, 1987.

[Woolf, Blegen, Jansen and Verloop, 1986] Woolf B.P., Blegen, D.,
Jansen, J.H., and Verloop, A., Teaching a Complex Industrial
Process , Proceedings of the National Conference on Artificial
Intelligence, Vol. II, Austin, TX, 1986, pp. 722-728.

[Yazdani, 1986] Yazdani, M. "Intelligent Tutoring Systems Survey,"
Artificial Intelligence Review, Vol. 1, p. 43, 1986.

433

Figure 1 . A Schematic Diagram of the General Architecture

Task Trial

0 1 23 456 7891 01 11 21 3

Taak Trial

Figure 2 .

Performance of Novices Using the PD/ICAT System

434